Testing Task Force Quarterly Update

This article originally appeared in The Bar Examiner print edition, Summer 2019 (Vol. 87, No. 2), pp 40–42.

For the latest Testing Task Force news—and to sign up to receive updates with the latest Task Force news, research, and blog posts—visit www.testingtaskforce.org

In January 2018, the National Conference of Bar Examiners appointed a Testing Task Force charged with undertaking a three-year study to ensure that the bar examination continues to test the knowledge, skills, and abilities (KSAs) required for competent entry-level legal practice in the 21st century. The study is scheduled to be completed by the end of 2020.

The Testing Task Force’s study will be

Comprehensive. The study will identify core competencies for newly licensed lawyers and explore when and how those competencies should be assessed.
Future-focused. The study will help ensure that the assessment of core legal competencies keeps pace with a rapidly evolving legal profession.
Collaborative. The Task Force will solicit input from state bar admitting authorities, legal educators, and legal professionals, recognizing a shared objective of ensuring the protection of the public and the readiness of new lawyers to join the profession.
Empirical. The study will be supported by research and review conducted by independent professionals with relevant technical expertise.
Transparent. The Task Force will welcome feedback and share updates on its website as the study progresses.

The beginning of 2018 has been a busy and exciting time for the Testing Task Force. Members of the Task Force, together with internal and external experts, are currently reviewing research proposals from independent consulting organizations; the Task Force expects to select a consultant and finalize a research plan soon.

Since their appointment in January, Task Force members have also been gathering information and reviewing current research on

Legal education, employment in the field, and trends in the profession;
Licensure and certification models in other professions;
Methodologies to best determine KSAs;
Best practices in conducting job/practice analyses; and
how other countries assess competency for admission to the practice of law.

Testing Task Force FAQs

Why is NCBE studying the bar exam?

NCBE studies many aspects of the bar examination on an ongoing basis. Periodic confirmation that the content being tested on a licensing exam aligns with competencies expected of a newly licensed professional is common and prudent in the field of licensing exams. The last large-scale survey-based study of this nature undertaken by NCBE was conducted via a job analysis in 2012. The Testing Task Force reflects NCBE’s ongoing efforts to support jurisdictions and to sustain confidence by all stakeholders in the testing process.

How is the Task Force study different from the 2012 job analysis?

The Task Force study will be more comprehensive than the 2012 job analysis in several ways. The 2012 study identified the job activities of newly licensed lawyers in sufficient detail to provide a job-related and valid basis for the development of licensing examinations offered by NCBE. The Task Force study will identify competencies required of newly licensed lawyers in the 21st century; it will also research whether, how, and when the identified competencies could be assessed and in the process will study test formats, test delivery methods, and the timing of test administration.

Why is a more comprehensive study required now?

The legal profession, like other professions, is undergoing a period of accelerated change. Some of the changes are brought on by rapid developments in technology that impact both the delivery of legal services and the types of legal problems for which clients seek representation. Technology is also contributing to advances in the science of assessment and testing. In addition, competency modeling in the legal profession has become increasingly important in a variety of contexts, including assessment of competencies expected of newly licensed lawyers to protect the public. It is prudent to undertake a comprehensive and systematic study at this time to ensure that the bar exam keeps pace with developments and changes in the practice of law, legal education, technology, and testing.

How will the Task Force solicit input from jurisdictions and other stakeholders?

The Task Force will soon retain an independent research consultant to outline a research plan for the study. Although that consultant has not yet been selected, the Task Force has informed potential consultants of the critical importance of incorporating jurisdiction and stakeholder input into the research plan. We anticipate that the study’s research methodology will gather information from our stakeholders through focus groups, forums, and surveys. In addition, the Task Force will actively solicit input from stakeholders through its website.

Who are the members of the Task Force?

Members of the Task Force bring many years of combined experience from their roles serving on the bench, in private practice, on state boards of bar examiners, as bar administrators, as legal employers, and as legal educators. They are familiar with NCBE’s examinations and knowledgeable about its test development processes. The Task Force is purposefully small in number to permit deliberate execution of its work, but it is committed to a comprehensive and collaborative study that draws heavily on independent expert assistance and on outreach to and input from stakeholders. See the Task Force members at www.testingtaskforce.org.

What is the role of the Task Force?

The Task Force will oversee the study, which will include gathering input from stakeholders. The Task Force will periodically report on its progress to the NCBE Board of Trustees and through updates on its website. Ultimately, the Task Force will prepare a final report setting out its recommendations. The report will provide a documented record of the Task Force’s work and will include qualitative and quantitative data that support its findings and recommendations.

When will the study be completed?

The Task Force plans to complete its work and submit its report to the NCBE Board of Trustees by the end of 2020.

Is the bar exam going to change?

The Task Force is approaching this study with no preconceived conclusions about whether the bar exam will or should change, whether in terms of content, format, timing, or method of delivery. Recommendations made by the Task Force will be driven by empirical research and stakeholder input. If any changes to the bar examination are recommended by the Task Force, all stakeholders, particularly state bar admission authorities and legal educators, will be informed about the nature of and the rationale for the proposed changes and will be given a meaningful opportunity to provide input on those changes. As has always been the case, no changes will be made to the bar exam without affording ample notice.

Will the Task Force study affect the Uniform Bar Exam?

The Uniform Bar Exam (UBE) is not a test product. Rather, it is a compact among jurisdictions that agree to use the same exam components, to adopt a set of policies to produce portable bar exam scores, and to accept the transfer of those scores from other UBE jurisdictions. The Task Force’s study will not change the premise on which the UBE is based—that use of the same exam components, administered and graded uniformly, produces bar exam scores that can be relied upon by other jurisdictions, sparing applicants from sitting for the bar exam in each jurisdiction in which they seek admission. UBE jurisdictions are stakeholders whose input will be sought and encouraged as the Task Force undertakes its study.

Will the study include a standard-setting study?

Setting the passing score for licensure is the prerogative of each jurisdiction. For that reason, the Task Force does not plan to include a standard-setting study within the scope of its work. However, research on score conversion or similar topics may be conducted if the Task Force’s study warrants it. NCBE will continue to provide jurisdictions with guidance regarding the appropriate interpretation and use of test scores.

Other groups are studying the bar exam now, too. Will the Task Force work with these groups?

The Task Force looks forward to receiving feedback and input from other stakeholder groups that are studying the bar exam. Although the Task Force study will be undertaken independently, the Task Force plans to share the results of its research with other interested stakeholders.